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[57] ABSTRACT 

This invention is a novel high-speed neural network 
based processor for solving the “traveling salesman” 
and other global optimization problems. It comprises a 
novel hybrid architecture employing a binary synaptic 
array whose embodiment incorporates the fixed rules of 
the problem, such as the number of cities to be visited. 
The array is prompted by analog voltages representing 
variables such as distances. The processor incorporates 
two interconnected feedback networks, each of which 
solves part of the problem independently and simulta- 
neously, yet which exchange information dynamically. 

29 Claims, 6 Drawing Sheets 


AB AC AD BA BC BD CA CB CD DA DB DC 




U.S. Patent 


Oct. 19, 1993 


Sheet 1 of 6 


5 , 255,349 



t 



•ab 'ac 'ad 'ba 'bc 'bd 'ca 'cb >cd 'da >db 'DC 


U.S. Patent 


Oct. 19, 1993 


Sheet 2 of 6 


5,255,349 



o 


FIG. 



U.S. Patent 


Oct. 19, 1993 


Sheet 3 of 6 


5,255,349 



RSN 


ROW 

SUPER 

NEURON 


CSN 


COLUMN 

SUPER 

NEURON 


IgSN 

GLOBAL 

SUPER 

NEURON 



FIG. 4 














U.S. Patent 


Oct. 19, 1993 


Sheet 4 of 6 


5,255,349 



FIG. 5 


















U.S. Patent 


Oct. 19, 1993 


Sheet 5 of 6 


5,255,349 



FIG. 6 








U.S. Patent 


Oct. 19, 1993 


Sheet 6 of 6 


5,255,349 




FIG. 9 


5,255. 


ELECTRONIC NEURAL NETWORK FOR 
SOLVING TRAVELING SALESMAN” AND 
SIMILAR GLOBAL OPTIMIZATION PROBLEMS 

5 

ORIGIN OF THE INVENTION 

The invention described herein was made in the per- 
formance of work under a NASA contract, and is sub- 
ject to the provisions of Public Law 96-517 (35 USC 
202) in which the Contractor has elected not to retain 10 
title. 

This is a continuation-in-part application of U.S. pa- 
tent application Ser. No. 07/470,664 filed Jan. 26, 1990 
and now is abandoned. 

15 

TECHNICAL FIELD 

The invention relates to neural computing networks 
and, more particularly, to a high-speed neural network 
based processor for solving global optimization prob- 
lems involving selecting a single optimum route be- 20 
tween a plurality of points comprising, a binary synaptic 
array incorporating the fixed rules of the problem being 
solved and having a plurality of inputs thereto and out- 
puts therefrom, the outputs comprising the answer to a 
problem being solved by the processor; neural connect- 25 
ing means comprising a plurality of neurons each hav- 
ing an input and an output for connecting inputs into the 
binary synaptic array, the outputs of the neurons being 
connected to the inputs of the binary synaptic array; 
analog prompting means connected to the inputs of the 30 
neurons for prompting the array with analog voltages 
representing variables of the problem being solved; and, 
feedback means for feeding back outputs from the bi- 
nary synaptic array to the neural connecting means to 
cause the processor to precipitate an answer to a prob- 35 
lem being solved, the feedback means comprising two 
interconnected feedback networks, each of which 
solves part of the problem being solved independently 
and simultaneously, the two interconnected feedback 
networks being connected to exchange information 40 
dynamically, a first one of the two interconnected feed- 
back networks solving a basic part of the problem being 
solved and a second one of the two interconnected 
feedback networks monitoring the solution to the basic 
part of the problem being solved by the first one and 45 
adding further limitations to assure that the first one 
provides one and only one solution to the basic part of 
the problem being solved which meets all predefined 
constraints defining the entire problem being solved, 
the first one of the two interconnected feedback net- 50 
works employing binary inhibitory synapses in a feed- 
back matrix to enforce constraints associated with the 
problem to be solved which are binary in nature and 
feeds analog conditions into the first one of the two 
interconnected feedback networks using analog 55 
prompting whereby a user can dynamically change 
analog represented variables of the problem to be 
solved, the binary inhibitory synapses including means 
for grouping the neurons into groups and for preventing 
more than one of the neurons in a group from indicating 60 
travel between a pair of the plurality of points at any 
given time, the second one of the two interconnected 
feedback networks including, a plurality of first voltage 
sources representing respective ones of the plurality of 
points, a plurality of binary switch means connected to 65 
respective ones of the plurality of voltage sources for 
providing a feedback control signal to the first one of 
the two interconnected feedback networks, each of the 
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plurality of binary switch means being associated with 
one of the neurons and being connected to receive a 
control signal therefrom whereby the plurality of bi- 
nary switch means are turned “ON” and “OFF” by 
their associated one of the neurons, a single output 
connected to the plurality of binary switch means for 
outputting a voltage which is the sum of the voltages 
from the plurality of voltage sources connected to the 
output by ones of the binary switch means which have 
been turned “ON” by an associated one of the neurons, 
and differential sensing means having a first input con- 
nected to the single output and a second input con- 
nected to a second voltage source equal to the sum of 
the plurality of first voltage sources for outputting a 
RESTART feedback control signal to the first one of 
the two interconnected feedback networks when the 
differential sending means finds that voltage values at 
the first and second inputs of the differential sensing 
means are not equal. 

BACKGROUND ART 

In the field of computing hardware, there is a class of 
global optimization problems which is typified by the 
so-called “traveling salesman problem” (TSP). In the 
TSP, a salesman with N cities on his itinerary is to select 
the round-trip tour that goes through each of the N 
cities and minimizes the total distance travelled. As 
those skilled in the art are well aware, such “global 
optimization problems” are very computation intensive 
since the time required to find the optimal tour solution 
by an exhaustive search grows faster than any polyno- 
mial function as the size of the problem increases. 

In the past, a variety of heuristic search algorithms 
have been applied to this problem with varying degrees 
of efficiency. Artificial neural network approaches to 
TSP have also been proposed in recent years (e.g. Hop- 
field and Tank in 1985). These prior art neural network 
approaches to TSP typically require a large number 
2N 3 for N cities of analog synapses. Such prior art neu- 
ral network solutions have been tested on software 
digital simulators; however, their hardware implemen- 
tation as an actual neural network have seldom been 
attempted due primarily to the involved hardware com- 
plexity in the artwork architectures. 

Multiparameter optimization problems impose multi- 
ple constraints, to be dealt with simultaneously, to se- 
lect the “best situation” or the global solution under 
given conditions. With increasing dimensionality, there- 
fore, solutions of such problems become highly compu- 
tation intensive. A brute force approach in a complex 
situation requires actual comparison and search through 
all possible solutions to determine and select the “lowest 
cost” situation. In the case of the TSP, for example, for 
the N cities the possible number of distinct tours is given 
by T=(N)!/2N. For four cities, the number of tours is 
only three. For only eight cities, however, the number 
grows to 2,520. At only twelve cities, the number of 
tours leaps to almost 20 million. Clearly, such problems 
quickly approach a level where a brute force approach 
is impractical even at modem computer speeds. In por- 
table implementations, for example, where computer 
power is very limited, a better solution is obviously 
needed. 

The above-mentioned neural network solution of the 
TSP proposed by Hopfield and Tank consists of an 
NxN array of neurons. The rows are labelled by the 
cities and the columns by the order in which the cities 
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are visited. Thus, a valid tour corresponds to having 
only a single active neuron in each row as well as in 
each column of the neuron array. Lateral inhibition 
along the rows as well as columns is required to enforce 
these constraints, in addition to a large number (~2N 3 ) 5 
of analog synapses to ‘‘store” the inter-city distance 
information. Although the algorithm does capture all 
the constraints in the proposed network, the complexity 
of the network architecture has been the primary con- 
straint in preventing its actual hardware implementa- 10 
tion. In other words, it works in simulation; but, is too 
complex to build in actuality. The promise of high speed 
solutions to TSP and similar global optimization prob- 
lems from the neural net approach, therefore, has not 
been realized so far. 15 

STATEMENT OF THE INVENTION 

Accordingly, it is an object of this invention to pro- 
vide an implementation of the traveling salesman prob- 
lem in a neural network that lends itself to easy hard- 20 
ware implementation. 

It is another object of this invention to provide an 
approach to the implementation in a neural network of 
a general class of global optimization problems as typi- 
fied by the traveling salesman problem. 25 

Other objects and benefits of this invention will be- 
come apparent from the detailed description which 
follows hereinafter when taken in conjunction with the 
drawing figures which accompany it. 

BRIEF DESCRIPTION OF THE DRAWINGS 30 

FIG. 1 is a simplified drawing depicting the first 
network portion of the present invention. 

FIG. 2 is a simplified drawing depicting how the first 
network portion of FIG. 1 may provide a solution to the 35 
problem to be solved not meeting all the constraints of 
the total problem. 

FIG. 3 is a simplified drawing depicting the auxiliary 
network portion of the present invention which is used 
in combination with the first portion of FIG. 1 to pre- 40 
vent the situation shown in FIG. 2. 

FIG. 4 is a simplified schematic block diagram of a 
neural network in accordance with an alternative em- 
bodiment of the invention. 

FIG. 5 is a simplified schematic block diagram of a 45 
portion of a cluster buster circuit corresponding to an 
improvement of the embodiment of FIG. 3. 

FIG. 6 is a simplified schematic block diagram illus- 
trating a portion of the circuit of FIG. 5 embodying a 
set of loop detectors. 50 

FIG. 7 is a simplified schematic diagram of a loop 
detector circuit corresponding to FIG. 6. 

FIG. 8 is a simplified schematic diagram illustrating a 
portion of the circuit of FIG. 5 embodying a set of 
vertex detectors forming a vertex detector network. 55 

FIG. 9 is a simplified schematic diagram illustrating a 
control circuit responsive to the loop and vertex detec- 
tors of FIGS. 6 and 9. 

DETAILED DESCRIPTION OF THE ^ 
INVENTION 

The present invention as now to be described 
achieves its stated objectives by employing a hybrid 
approach in which analog prompting is utilized in asso- 
ciation with only binary synapses. This, of course, pro- 65 
vides an implementation which is significantly simpler 
to implement in hardware than the traditional wholly 
analog approach of the prior art. In the approach taken 
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by the inventors herein, appropriate partitioning of the 
basic problem of TSP and the unique scheme of analog 
prompting make this significant difference in the overall 
complexity of the network. Constraints such as (1) visit 
every city only once, (2) do not retrace the path, and (3) 
do not skip any city, are essentially binary in nature. On 
the other hand, the requirement to minimize the total 
distance travelled clearly deals with analog quantities. 

This invention uses binary inhibitory synapses in a 
feedback matrix to enforce those constraints which are 
binary in nature and feeds analog conditions into the 
network using analog prompting. The major difference 
from prior art solutions to TSP, for example, lies in the 
representation of the problem in the network. The neu- 
rons in this invention represent inter-city connections 
rather than the cities and the sequence in the tour. This 
allows the user to represent (i.e., feed in) the analog 
distances or costs of the corresponding travels as analog 
prompts, or fixed bias inputs, to the neurons in the net- 
work. 

FIG. 1 shows a schematic of a feedback network 10 
according to the present invention implementing a four- 
city (A, B, C, and D) TSP. The neurons 12 are associ- 
ated with the inter-city travels (e.g. AB, AC, AD, BC, 
BD, ... etc.). The sparsely connected feedback matrix of 
synapses 14 represents the binary constraints. For exam- 
ple, when neuron 12 labelled “AB” is “ON” (i.e., the 
salesman travels from A to B), he cannot travel from A 
to C, A to D, or B to A in that tour; therefore, neuron 
12 ‘‘AB” inhibits, for example, “AC”, “AD”, and 
“BA”, through the inhibitory binary synapses 14 in the 
matrix. All such binary connections form the ‘‘symmet- 
ric” feedback matrix shown for four cities in FIG. 1. 
Therefore, in each subgroup of neurons 12 (e.g., AB, 
AC, and AD), lateral excitation/inhibition control is 
provided, as represented by the dashed boxes 16, such 
that one and only one neuron 12 among them is “ON”, 
since salesman can go “out” from city A only once. For 
a given situation, the inter-city distances are used to 
generate the analog excitatory prompts at 18 to all the 
neurons 12 (proportional to the corresponding distance- 
costs subtracted from a fixed, preselected excitation 
level). 

The network 10 of FIG. 1, when prompted at 18 with 
the appropriate analog values, does generate a tour that 
goes through all the cities, visiting each city only once 
and without skipping any city; however, it must be 
pointed out that in an attempt to minimize the distance 
cost as much as possible, this particular network often 
tends to generate smaller disjointed closed loops con- 
necting groups of cities independently, rather than tak- 
ing a single closed loop, particularly when the cities are 
clustered in groups farther away from each other. FIG. 
2 depicts such a solution for a six city TSP with two 
closed triangular loops 20 connecting three cities 22 
each (i.e., A, B & C and D, E & F) instead of the desired 
single six city tour. Of course, in that process the net- 
work 10 is doing a superb job of enforcing all the imple- 
mented constraints and minimizing the total cost. To 
enforce the remaining condition that there should be 
only one closed loop tour, therefore, an auxiliary net- 
work working in combination with the network 10 is 
necessary. FIG. 3 shows a schematic of the auxiliary 
network 24 for use in association with the above- 
described example of a four city TSP situation depicted 
in FIG. 1. The 4x4 array of binary switches 26, except 
for the diagonal, is directly connected to the output 
activity (with a fixed threshold) of all the neurons 12 of 
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the previous network 10, as labelled. The voltage 
sources 28 represent the cities and are labelled accord- 
ingly (e.g., A, B, C and D). When a solution starts 
emerging from the first network 10 of FIG. 1, the 
switches 26 in the auxiliary network 24 close and the 5 
signals from the different city (i.e., voltage) sources 28 
start connecting themselves in a series, adding to a sig- 
nal that goes back to the first network 10. In one possi- 
ble scenario, the auxiliary network 24 can be used to 
inhibit several of the set of neurons 12 which are “ON” 10 
in a multi-loop solution. This forbids the multi-loop 
solution (e.g., as depicted in FIG. 2) from being a stable 
state of the network 10. On the other hand, if all cities 
are connected in a single tour as desired, the auxiliary 
circuit 24 gives its “approval” by sending no inhibitory 15 
signal to the winning set of neurons 12. 

The foregoing operation of the auxiliary network 24 
of FIG. 3 works as follows. Each city’s voltage source 
28 provides a fixed voltage potential (V). The output 30 
of the auxiliary network 24 is connected as one input of 20 
a differential amplifier 32. The other input of the differ- 
ential amplifier 32 is connected to a voltage source 34 
which is equal to the sum of the Vs for all the cities. In 
this case, therefore, with four cities the voltage source 
34 provides a constant voltage of 4V to the differential 25 
amplifier 32. The output of the differential amplifier 32 
is connected to thresholding logic 36. If the threshold- 
ing logic 36 senses that the output of the differential 
amplifier 32 is zero, it knows that all the cities are in- 
cluded in the present solution since there is a connection 30 
through the switches 26 (controlled by the connections 
to the network 10 of FIG. 1) to each voltage source 28 
(in order for 4V to be output at 30). In such a case, the 
thresholding logic 36 outputs a “SOLUTION” indica- 
tion as indicated. If, on the other hand, the thresholding 35 
logic 36 senses that the differential amplifier 32 is out- 
putting other than zero, it knows that all cities are not 
accounted for in the present solution being proposed by 
the network 10 and, therefore, the thresholding logic 36 
outputs a “RESTART” command to the network 10 of 40 
FIG. 1. Thus, the auxiliary network 24 of FIG. 3 in 
combination with the network 10 of FIG. 1 assures that 
the proper total solution to the TSP is quickly precipi- 
tated. 

In another embodiment, the large number 0(N 3 ) of 45 
binary /analog synapses is replaced by only 2N 2 binary 
synapses and analog synapses. This alternative 
embodiment is distinguished by the addition of 2N + 1 
supemeurons: one for each row, one for each column, 
and one for global control. In comparison with the 50 
previous method, this approach reduces the hardware 
complexity at least by a factor of N. As shown in FIG. 

4, the outputs of the neurons 38 from each row (column) 
are fed into the corresponding row- (column-) super- 
neuron, 40 (42) which in turn sends back an appropriate 55 
level of excitatory or inhibitory signal to all the neurons 
in that row (column), to ascertain a valid tour with a 
fixed number of “ON” neurons 38. The global super- 
neuron 44 similarly keeps a “watch” on the total num- 
ber of “ON” neurons in the selected solution. The 60 
supemeurons 40, 42, 44 therefore collectively provide 
the simultaneous dynamic control over the evolution of 
the circuit of FIG. 4 during its convergence behavior. 

In our network, the usual input resistor rho and input 
capacitor C (used typically in a fully connected feed- 65 
back neural network) for each neuron are replaced by 
simple low pass filter circuits using an “RC” circuit (not 
shown). Furthermore, the usual current bias is also not 
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necessary as the supemeurons 40, 42, 44 can be very 
sensitive for taking over the role of bias currents in 
feedback networks. Our software simulation results 
show that a single loop or a multi-loop solution is al- 
ways found by our circuit with supemeurons with guar- 
antee. Further, the cluster buster circuit described 
below is simple enough to implement in analog hard- 
ware and promises to find a single loop solution in a 
very short time. 

DESCRIPTION AND EXPLANATION 

The proposed feedback network (FIG. 4 ) utilizes 
three types of neurons: 

1. Nx(N-l) sigmoid neurons 38 associated with 
individual inter-city travels. These are arranged in a 
“diagonal-less” NxN array and are of the type em- 
ployed in the embodiment of FIG. 1. As indicated in the 
legend of FIG. 4, each neuron 38 responds to two inputs 
Ai and Aj, which are analog voltages corresponding to 
distances or transitions between cities or points. Each 
neuron 38 generates an output signal which is a sigmoid 
function of the two inputs Ai and Aj. 

2. Local row and column supemeurons 40, 42 (RSN 
and CSN) associated with each row (and column) of the 
sigmoid neurons 38 described above. There are of 
course 2N of these for the N-city problem. Each of 
these superneurons 40, 42 only control the correspond- 
ing group of neurons in a single row (or column) to 
satisfy the local constraint (e.g., only two “ON” sig- 
moid neurons 38 in a given row or column if N=4). 

3. A single “global” supemeuron 44 (GSN) that mon- 
itors the total activity of all the N(N-l) sigmoid neu- 
rons, 40, 42 to meet the requirement of the constraint 
that only N “inter-city travels” are allowed globally. 

Sigmoid neurons 38 selected as the final solution must 
therefore satisfy the constraints imposed by all the 
supemeurons 40, 42, 44 simultaneously, in order to 
obtain a valid solution. The symmetry across the diago- 
nal in the array (FIG. 4) dictates that, each of the N row 
supemeurons 40 (or each of the N column supemeurons 
42) must ensure two and only two sigmoid neurons 
“ON” in each row (column) in the case of N=4, for 
example. This is the local constraint which implements 
the ultimate constraint that every city is visited once 
and only once in a closed tour. 

Since the “diagonal-less” matrix of the sigmoid neu- 
rons 38 of FIG. 4 is symmetrical across the (missing) 
diagonal, the tour can be read directly from only the 
upper triangle or the lower triangle of the matrix. The 
duplication of the triangle is only an implementation 
convenience. After providing all the costs of the inter- 
city travels to the matrix of sigmoid neurons as analog 
prompts in the manner of the embodiment of FIG. 1, the 
matrix of FIG. 4 goes into action. To start with, several 
of the low cost sigmoid neurons 38 tend to come “ON”. 
If the tour however is not complete, supemeurons 40, 
42 act based on the shortage or excess in the number of 
the “ON” neurons in each row and column, providing 
the desired excitation or inhibition, respectively. The 
procedure occurs simultaneously at every neuron. 
Since, in this process, it is difficult for the primary deci- 
sion to interchange information between rows and col- 
umns while interacting with high gain neurons, the 
global neuron 44 is needed to provoke and bring back 
the process in a reasonable linear region of the dynamics 
in order to exchange the information globally. The 
global supemeuron 44 thus checks the 2N sigmoid neu- 
rons which are “ON” in the whole network. The inhibi- 
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tion or excitation is applied by the global supemeuron 
44 to all the sigmoid neurons 38 depending on the excess 
or shortage in total number of neurons which are “ON”. 

To understand the dynamics of a complete conver- 
gence cycle of the network of FIG. 4, consider a set of 5 
analog prompts being applied to the array of sigmoid 
neurons 38. All sigmoid neuron outputs are initially 
low. All RSN’s 40, CSN’s 42, and the GSN 44 that do 
not get sufficient input from the sigmoid neurons 38 
generate excitations for their corresponding rows and 10 
columns. As the analog prompt to each sigmoid neuron 
38 provides varying activity in the array, some of the 
sigmoid neurons 38 increase their output faster than 
others and the associated RSNs 40 and CSNs 42 adjust 
their excitatory /inhibitory feedback levels to dynami- 15 
cally satisfy the given constraints, until the solution is 
reached. For example, let the sigmoid neuron AB (and 
thus BA) be on. The row and column that include those 
neurons AB and BA would still not be satisfied with 
their constraints (two sigmoid neurons “ON” per row 20 
and column). The excitations are thus applied to stimu- 
late other neurons such as AC, AD, BC, and BD, etc. 
Although this example is explained in a serial fashion, it 
should be noted all the components of the network in an 
implementation would actually behave in a fully paral- 25 
lei fashion. 

Unfortunately, the network of FIG. 4 also does not 
guarantee a single loop tour. Depending on the posi- 
tions and distribution of the cities, a multi-loop tour is 
occasionally formed. This is not a negative attribute of 30 
the network at all because the network in fact does 
attempt to always provide the lowest cost solution as it 
should, which only happens to be a multi-loop solution 
in some cases, due to the city distribution. Thus, the 
network provides an additional piece of valuable infor- 35 
mation: “hire more than one salesman”. 

FIG. 2 discussed above shows a low cost multi-loop 
solution for N = 6 with two closed triangle loops, each 
of which is connecting three cities, instead of a single six 
city tour, which would of course be a higher cost solu- 40 
tion. To reduce the multi-loop solution to a single loop, 
a modified cluster buster circuit is proposed in the fol- 
lowing. The following cluster buster circuit (FIG. 5) is 
useful with either of the embodiments of FIG’S. 1 and 4. 

In order to achieve a fast response from the cluster 45 
buster circuit, N(N — 1) sigmoid neuron outputs are 
made to communicate with the cluster buster circuit in 
a parallel fashion. The matrix of FIG. 4 of the sigmoid 
neurons 38 is symmetrical. Therefore, we need to use 
only N(N — 1) /2 (upper triangle or lower triangle ma- 50 
trix) outputs to give enough information to the cluster 
buster circuit. However, in the current version of the 
cluster buster circuit (schematically shown in FIG. 5) a 
complete, symmetrical matrix of a cluster buster circuit 
is used. Based on the provided information, the cluster 55 
buster circuit of FIG. 5 detects the presence of a multi- 
loop situation and generates corrective action to push 
the network of FIG. 4 towards finding a valid single 
loop solution. In a nutshell, when the circuit of FIG. 5 
detects multiple closed loops, it inputs additional 60 
“costs” to selected neurons 38 forming the multi-loop 
solution of FIG. 4 (in effect expanding the correspond- 
ing travel distances involved in those loops). These 
costs increase gradually until the closed loops are bro- 
ken, leading to a convergence of the sigmoid neuron 65 
array of FIG. 4 in a different direction. The cluster 
buster circuit of FIG. 5 therefore consists of two sub- 
circuits at each node 46: a loop detector circuit 48 and 
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a vertex detector circuit 50. Each of these circuits 48, 50 
functions independently but in close cooperation with 
each other. Both loop and vertex detector circuits 48, 50 
must be “ON” simultaneously, to activate the action of 
breaking the loops. The “ON” cells corresponding to 
the solution of FIG. 2 are shaded in FIG. 5. 

Each node 46 as shown in FIG. 5 has two parts: the 
upper rectangle represents a loop detector 48 that 
checks if the corresponding city is contained in any 
loop, and the lower rectangle represents a vertex detec- 
tor 50. All of the vertex detectors 50 in FIG. 5 together 
form a vertex detector network to be described below 
which detects the presence of a multi-loop solution. For 
the loop detector 48 (shown separately for clarity in 
FIG. 6), a simple current gain amplifier cell is used. 
There are two parts 52, 54 in this cell 48 (see FIG. 7). A 
constant current source i, if the corresponding sigmoid 
neuron 38 is “ON”, provides an output current from the 
first part of the cell 52 to the overall output line of each 
row of the cluster buster circuit of FIG. 6, establishes a 
certain voltage level for that row. Secondly, the feed- 
back current collected from all the cities in the same 
row (say i th row) is fed into the feedback input of the 
second part 54 of the corresponding (i th) column with 
a gain. The symmetry of the matrix dictates that if any 
closed loop is formed (which invariably would need 
nodes from several rows and columns), the voltage 
level at the corresponding rows would increase contin- 
uously due to the constructive feedback, reaching 
above a certain predetermined threshold Vthresh. This 
would be a direct indication of a closed loop formation. 
If the travels selected at any time do not form a closed 
loop, then the corresponding row output voltage would 
decrease exponentially with feedback and would al- 
ways collapse consistently and remain below the thresh- 
old. Therefore, a comparator 56 in the second part 54 
produces an “ON” or “OFF” signal depending upon 
whether the feedback voltage is above or below 
Vthresh, respectively. 

Returning to FIG. 5, all the vertex detectors 50 form 
a network partially shown in FIG. 8. A current mirror 
58 is arbitrarily assigned to one row of the vertex detec- 
tor network. In the vertex detector network of FIG. 8, 
a current source 60 is assigned for every row corre- 
sponding to a city. (The rows and columns of the vertex 
detector network of FIG. 8 correspond to the rows and 
columns of the neural network of FIG. 1 or FIG. 4. It 
will be remembered that, in accordance with the em- 
bodiment of FIG. 1, each neuron 38 corresponds to an 
inter-city trip while each row or column corresponds to 
a city.) In addition, all cities communicate pairwise with 
each other in the vertex detector network of FIG. 9 
through a set of switches (transistors) 62, comprising 
each vertex detector 50 and having control inputs con- 
nected to outputs of corresponding sigmoid neurons 38, 
as indicated in FIG. 8. If a closed loop is formed, any 
“ON” sigmoid neurons 38 in the loop would turn on 
corresponding switches 62 to connect rows (cities) to- 
gether. If the current mirror 58 happens to be a part of 
a loop, it would reflect the number of cities in that loop 
by summing all the current sources 68. If the current 
output from the current mirror 58 is larger than (N— 3) 
Io, a comparator 64 provides a vertex detector “OFF” 
output. Otherwise, it generates an “ON” output. With 
the vertex detector “OFF” and the loop detector 
“ON”, we can imply that a single loop solution is found. 
If the current mirror 58 does not belong to any loops, 
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we can imply that the solution is not found. In this case, 
the vertex detector output is “ON”. 

In order to check for multi-loops and to bust the 
closed loops efficiently to each inter-city connection, an 
AND gate 66 in the controller of FIG. 9 checks 5 
whether: 

(a) The loop detector is “ON” for that inter-city or 
sigmoid neuron; 

(b) it is not a single loop solution with N vertices 

(vertex detector-ON). 1° 

The controller of FIG. 9 operates as follows: 

If the output of the AND gate 66 is HIGH, it turns on 
an N-channel gate 68 to send the amount of current 
from the current source 52 to the corresponding sig- 
moid neuron 38 as inhibitory feedback, to increase the 15 
distance of the correspond inter-city trip. The addi- 
tional “cost” between cities in the loops is increased 
continuously until closed loops no longer exist, then the 
added “cost” is removed leaving the original analog 
inter-city distance values. In this embodiment, the addi- 20 
tional “cost” is imposed on any “ON” sigmoid neurons 
38 in proportion to the population of the loops of which 
they are members. 

As mentioned above, the controller of FIG. 9 breaks ^ 
up multiple closed loops by imposing added costs on 1 
each “ON” neuron 38 in proportion to the number of 
neurons comprising the closed loop of which it is a 
member. However, in an alternative embodiment of the 
controller of FIG. 9, the added cost instead may be 
proportional to the original cost (inter-city distance) or 
analog value of the neuron. This would be accom- 
plished by deriving the current source input to the tran- 
sistor 68 from the initial analog voltage which first 
establishes the inter-city distance of the neuron, rather 
than the current source 52. In yet another alternative 
embodiment of the controller of FIG. 9, the imposed 
cost would be a uniform constant amount for all “ON” 
neurons. This would be accomplished by providing the 
same current source to all of the transistors 68. Finally, 4Q 
the controller of FIG. 9 may embody a combination of 
all of the foregoing ways of computing the added cost 
for each “ON” neuron. 

In a corollary variation of the controller of FIG. 9, 
rather than adding cost to the “ON” neurons in the 45 
form of inhibitory feedback, as discussed above, cost 
may be subtracted from the “OFF” neurons in the form 
of excitatory feedback. In order to accomplish this, the 
loop detector input to the AND gate 66 would be in- 
verted and the polarity of current through the transistor 50 
68 , as applied to the neuron analog input, would be 
inverted. In this way, and in accordance with the fore- 
going description, the added “reward” (as opposed to 
added “cost”) would be one of the following or combi- 
nation thereof: (a) inversely proportional to the number 55 
neurons in the loop including a city corresponding to 
the “OFF” neuron, (b) inversely proportional to the 
inter-city distance associated with the “OFF” neuron or 

(c) a uniform constant amount for all “OFF” neurons. 

Wherefore, having thus described our invention, 60 
what is claimed is: 

1. A high-speed neural network for solving global 
optimization problem involving selecting routing be- 
tween a plurality of points comprising: 

a binary synaptic array having a plurality of inputs 65 
and outputs and incorporating fixed rules of the 
problem being solved in a plurality of connecting 
binary inhibitory synapses; 
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b) a neuron array comprising a plurality of non-linear 
neuron elements each having an input and an out- 
put with said outputs of said non-linear neuron 
elements connected to respective ones of said in- 
puts of said binary synaptic array; 

c) analog prompting means connected to said inputs 
of said non-linear neuron elements for prompting 
said neuron array with analog voltages represent- 
ing variables of the problem being solved; and, 

d) feedback means connected from said outputs of 
said binary synaptic array to respective ones of said 
inputs of said non-linear neuron elements, said out- 
puts of said binary synaptic array also comprising 
the output of the neural network based processor. 

2. The neural network of claim 1 wherein: 

the network incorporates two interconnected feed- 
back networks, each of which solves part of the 
problem independently and simultaneously, said 
two interconnected feedback networks being con- 
nected to exchange information dynamically. 

3. The neural network of claim 2 wherein: 

a) a first one of said two interconnected feedback 
networks solves a part of the problem; and, 

b) a second one of said two interconnected feedback 
networks monitors solution of said part of the prob- 
lem and adds further limitations to assure that said 
first one provides one and only one solution to said 
part of the problem which meets all predefined 
constraints defining the problem being solved. 

4. The neural network of claim 3 wherein: 

said first one of said two interconnected feedback 
networks employs binary inhibitory synapses in a 
feedback matrix to enforce constraints associated 
with the problem to be solved which are binary in 
nature and feeds analog conditions into said first 
one of said two interconnected feedback networks 
using analog prompting whereby a user can dy- 
namically change analog represented variables of 
the problem to be solved. 

5 . The neural network of claim 4 wherein: 

said binary inhibitory synapses are disposed to be 
associated with a plurality of groups and include 
means for preventing more than one of said non- 
linear neuron elements of a group from being 
“ON” to indicate travel between a pair of the plu- 
rality of points at any given time. 

6. The neural network of claim 3 wherein said second 
one of said two interconnected feedback networks in- 
cludes: 

a) a plurality of first voltage sources representing 
respective ones of the plurality of points; 

b) a plurality of binary switch means connected to 
respective ones of said plurality of first voltage 
sources for providing a feedback control signal to 
said first one of said two interconnected feedback 
networks, each of said plurality of binary switch 
means being associated with one of said non-linear 
neuron elements and being connected to receive a 
control signal therefrom whereby said plurality of 
binary switch means are turned “ON” and “OFF” 
by their associated one of said non-linear neuron 
elements; 

c) an output connected to said plurality of binary 
switch means for outputting a voltage which is the 
sum of the voltages from said plurality of voltage 
sources connected to said output by ones of said 
binary switch means which have been turned 
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“ON” by an associated one of said non-linear neu- 
ron elements; and, 

d) differential sensing means having a first input con- 
nected to said output and a second input connected 
to a second voltage source equal to the sum of said 
plurality of first voltage sources for outputting a 
RESTART feedback control signal to said first one 
of said two interconnected feedback networks 
when said differential sensing means finds that 
voltage values at said first and second inputs of said 
differential sensing means are not equal. 

7. The neural network of claim 1 wherein: 

a) said binary inhibitory synapses are disposed to be 
associated with a plurality of groups; and, 

b) said non-linear neuron elements are interconnected 
in an inhibitory manner to prevent more than one 
of said non-linear neuron elements associated with 
a group from connecting inputs into said network 
to indicate travel between a pair of the plurality of 
points at any given time. 

8. A high-speed neural network for solving a global 
optimization problem involving selecting a single opti- 
mum route between a plurality of points comprising: 

a) a binary synaptic array incorporating fixed rules of 
the problem and having a plurality of inputs 
thereto and outputs therefrom, said outputs com- 
prising an answer to said problem; 

b) neural connecting means comprising a plurality of 
neurons each having an input and an output for 
connecting inputs into said binary synaptic array, 
said outputs of said neurons being connected to 
said inputs of said binary synaptic array; 

c) analog prompting means connected to said inputs 
of said neurons for prompting said binary synaptic 
array with analog voltages representing variables 
of the problem; and, 

d) feedback means for feeding back outputs from said 
binary synaptic array to said neural connecting 
means to precipitate an answer to said problem, 
said feedback means comprising two intercon- 
nected feedback networks, each of which solves 
part of the problem being solved independently 
and simultaneously, said two interconnected feed- 
back networks being connected to exchange infor- 
mation dynamically, a first one of said two inter- 
connected feedback networks generating a solution 
to a basic part of the problem and a second one of 
said two interconnected feedback networks moni- 
toring said solution to said basic part of the prob- 
lem and adding further limitations to assure that 
said first one provides one and only one solution to 
said basic part of the problem being solved which 
meets all predefined constraints defining the entire 
problem being solved. 

9. The neural network of claim 8 wherein: 

said first one of said two interconnected feedback 
networks employs binary inhibitory synapses in a 
feedback matrix to enforce constraints associated 
with the problem to be solved which are binary in 
nature and feeds analog conditions to said first one 
of said two interconnected feedback networks 
using analog prompting whereby a user can dy- 
namically change analog represent variables of the 
problem to be solved. 

10. The neural network of claim 9 wherein: 

said binary inhibitory synapses include means for 
grouping said neurons into groups and for prevent- 
ing more than one of said neurons in a group from 
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indicating travel between a pair of the plurality of 
points at any given time. 

11. The neural network of claim 8 wherein said sec- 
ond one of said two interconnected feedback networks 
includes: 

a) a plurality of first voltage sources representing 
respective ones of the plurality of points; 

b) a plurality of binary switch means connected to 
respective ones of said plurality of voltage sources 
for providing a feedback control signal to said first 
one of said two interconnected feedback networks, 
each of said plurality of binary switch means being 
associated with one of said neurons and being con- 
nected to receive a control signal therefrom 
whereby said plurality of binary switch means are 
turned “ON” and “OFF” by their associated one of 
said neurons: 

c) an output connected to said plurality of binary 
switch means for outputting a voltage which is the 
sum of the voltages from said plurality of voltage 
sources connected to said output by ones of said 
binary switch means which have been turned 
“ON” by an associated one of said neurons; and, 

d) differential sensing means having a first input con- 
nected to said single output and a second input 
connected to a second voltage source equal to the 
sum of said plurality of first voltage sources for 
outputting a RESTART feedback control signal to 
said first one of said two interconnected feedback 
networks when said differential sensing means 
finds that voltage values at said first and second 
inputs of said differential sensing means are not 
equal. 

12. The neural network of claim 8 wherein: 

said neural connecting means are interconnected in 
an inhibitory manner to prevent more than one of 
said neurons from connecting an input into said 
array to indicate travel between a pair of the plural- 
ity of points at any given time. 

13. A high-speed neural network for solving a global 
optimization problem involving selecting a single opti- 
mum route between a plurality of points comprising: 

a) a binary synaptic array incorporating fixed rules of 
the problem and having a plurality of inputs 
thereto and outputs therefrom, said outputs com- 
prising an answer to said problem; 

b) neural connecting means comprising a plurality of 
neurons each having an input and an output for 
connecting inputs into said binary synaptic array, 
said outputs of said neurons being connected to 
said inputs of said binary synaptic array; 

c) analog prompting means connected to said inputs 
of said neurons for prompting said binary synaptic 
array with analog voltages representing variables 
of the problem; and, 

d) feedback means for feeding back outputs from said 
binary synaptic array to said neural connecting 
means to precipitate an answer to said problem said 
feedback means comprising two interconnected 
feedback networks, each of which solves part of 
the problem independently and simultaneously, 
said two interconnected feedback networks being 
connected to exchange information dynamically, a 
first one of said two interconnected feedback net- 
works generating a solution to a basic part of the 
problem and a second one of said two intercon- 
nected feedback networks monitoring the solution 
to said basic part of the problem and adding further 
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limitations to assure that said first one provides one 
and only one solution to said basic part of the prob- 
lem which meets all predefined constraints defining 
the problem said first one of said two intercon- 
nected feedback networks employing binary inhibi- 
tory synapses in a feedback matrix to enforce con- 
straints associated with the problem [to be solved] 
which are binary in nature and feeds analog condi- 
tions into said first one of said two interconnected 
feedback networks using analog prompting 
whereby a user can dynamically change analog 
represented variables of the problem, said binary 
inhibitory synapses including means for grouping 
said neurons into groups and for preventing more 
than one of said neurons in a group from indicating 
travel between a pair of the plurality of points at 
any given time, said second one of said two inter- 
connected feedback networks including: 
dl) a plurality of first voltage sources representing 
respective ones of the plurality of points, 
d2) a plurality of binary switch means connected to 
respective ones of said plurality of first voltage 
sources for providing a feedback control signal to 
said first one of said two interconnected feedback 
networks, each of said plurality of binary switch 
means being associated with one of said neurons 
and being connected to receive a control signal 
therefrom whereby said plurality of binary switch 
means are turned “ON” and “OFF” by their asso- 
ciated one of said neurons, 
d3) a single output connected to said plurality of 
binary switch means for outputting a voltage 
which is the sum of the voltages from said plurality 
of first voltage sources connected to said output by 
ones of said binary switch means which have been 
turned “ON” by an associated one of said neurons, 
and 

d4) differential sensing means having a first input 
connected to said single output and a second input 
connected to a second voltage source equal to the 
sum of said plurality of first voltage sources for 
outputting a RESTART feedback control signal to 
said first one of said two interconnected feedback 
networks when said differential sensing means 
finds that voltage values at said first and second 
inputs of said differential sensing means are not 
equal 

14. A neural network for solving global optimization 
problems comprising: 

an array of neurons arranged in rows and columns, 
said neurons being associated with corresponding 
analog values characteristic of transitions between 
respective pairs of points, each of said neurons 
comprising means for outputting the correspond- 
ing analog value in an “ON” state and for output- 
ting a lesser value in an “OFF” state; 
a column super-neuron associated with each of said 
columns, said column super-neuron comprising 
means for applying feedback to each neuron in the 
corresponding column in accordance with the 
number of “ON” neurons in said column; 
a row super-neuron associated with each of said 
rows, said row super-neuron comprising means for 
applying feedback to each neuron in the corre- 
sponding row in accordance with the number of 
“ON” neurons in said row. 

15. The neural network of claim 14 further compris- 
ing: 
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a global super-neuron associated with all of said neu- 
rons, said global super-neuron comprising means 
for applying feedback to all of said neurons in ac- 
cordance with the number of “ON” neurons in said 
5 array. 

16. The neural network of claim 15 wherein: 

said global super-neuron comprises means for apply- 
ing an excitory feedback to the neurons in said 
array whenever the number “ON” neurons in said 
10 array is less than a global constraint: applying in- 
hibitive feedback thereto whenever the number of 
“ON” neurons in said array is greater than said 
global constraint: and applying no feedback when 
the number of “ON” neurons is equal to said global 
15 constraint. 

17. The neural network of claim 14 wherein: 

. said row super-neuron comprises means for applying 
an excitory feedback to the neurons in the corre- 
sponding row whenever the number “ON” neu- 
20 rons in said row is less than said the specified local 
constraint; applying the inhibitive feedback thereto 
whenever the number of “ON” neurons in said row 
is greater than said the specified local constraint 
and applying no feedback when the number of 

25 “ON” neurons is equal to the specified local con- 

straint; 

said global super-neuron comprises means for apply- 
ing an excitory feedback to the neurons in the cor- 
responding column whenever the number “ON” 
30 neurons in said column is less than said the speci- 
fied local constraint; applying the inhibitive feed- 
back thereto whenever the number of “ON” neu- 
rons in said column is greater than said the speci- 
fied local constraint and applying no feedback 
35 when the number of “ON” neurons is equal to the 
specified local constraint. 

18. The neural network of claim 17 wherein said rows 
and columns correspond to different points and said 
neurons correspond to transitions between pairs of said 

40 points and said analog values correspond to values of 
said transitions, and wherein said local and global con- 
straints limit the number of transitions between any pair 
of points to a single transition and participation of each 
said point is exactly two such transitions in the final 
45 stable state solution. 

19. The neural network of claim 18 wherein the num- 
ber of points is 4 and said local constraint requires said 
row and column super-neurons to apply inhibitive feed- 
back whenever the number of “ON” neurons in any 

50 row or column exceeds 2 and to apply excitory feed- 
back whenever the number of “ON” neurons in any 
row or column is less than 2. 

20. The neural network of claim 14 wherein each of 
said row and column super-neurons receives the outputs 

55 of the neurons in the corresponding row and column, 
respectively, and sends a feedback signal to first and 
second inputs of each neuron in said row and column, 
respectively. 

21. The neural network of claim 20 wherein said 
60 global super-neuron receives the outputs of all of said 

neurons and sends a feedback signal to a third input of 
each neuron. 

22. The neural network of claim 21 wherein each of 
said inputs is characterized by resistance which tends to 

65 have a stabilizing influence on said network. 

23. The neural network of claim 14 further compris- 
ing a circuit for breaking multiple loop solutions com- 
prising: 
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vertex detector means for sensing whenever there is 
more than one closed loop of said points corre- 
sponding to “ON” neurons; 

loop detector means for sensing the number of “ON” 
neurons in a closed loop; and 5 

means for transmitting feedback to selected ones of 
said neurons in response to said vertex and loop 
detector means. 

24. The neural network of claim 23 wherein said 
feedback transmitted by said means for transmitting is a 10 
function of the number of “ON” neurons in respective 
closed loops. 

25. The neural network of claim 24 wherein said 
means for transmitting feedback comprises means for 
transmitting an inhibitory feedback signal to said “ON” 15 
neurons proportional to the number of “ON” neurons 
included in the corresponding closed loop. 

26. The neural network of claim 24 wherein said 
means for transmitting feedback comprises means for 

20 
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transmitting an excitory feedback signal to an “OFF” 
neuron inversely proportional to the number of “ON” 
neurons included in a closed loop which includes a 
“ON” neuron having a point in common with said 
“OFF” neuron. 

27. The neural network of claim 23 wherein said 
means for transmitting transmits at least one of: 

(a) inhibitive feedback to selected ones of said “ON” 
neurons, and 

(b) excitory feedback to selected ones of said “OFF” 
neurons. 

28. The neural network of claim 27 wherein said 
feedback is one of (a) proportional and (b) inversely 
proportional to the corresponding analog value. 

29. The neural network of claim 27 wherein said 
feedback is a uniform amount for one of (a) said “ON” 

neurons and (b) said “OFF” neurons. 
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